System for detecting six degrees of freedom of movement by tracking optical flow of backscattered laser speckle patterns

ABSTRACT

Augmented reality headgear includes transparent displays that allow a user to simultaneously view the real world and virtual content positioned in the real world and further includes at least one source of coherent light and at least one sensor array for sensing, at a series of times, speckle patterns produced when the coherent light impinges environment surfaces. Circuitry is provided for sensing shifts in the speckle pattern and determining motion which caused the shift of the speckle pattern and adjusting the display of virtual objects displayed by the augmented reality headgear to compensate for the motion.

CROSS REFERENCE TO RELATED APPLICATION

This application is based on provisional patent application Ser. No.62/489,339 filed Apr. 24, 2017.

FIELD OF THE INVENTION

The invention pertains to augmented reality headgear.

BACKGROUND

Recently virtual reality devices that immerse a user in a computergenerated virtual world have been introduced. Certain virtual realitydevices include a pair of displays placed in close proximity to a user'seyes and corrective optics interposed between the pair of displays andthe user's eyes the purpose of which is to allow the user to focus onthe imagery displayed on the displays notwithstanding the closeproximity of the displays. The major application for such virtualreality devices is gaming although other applications such as scientificdata visualization are also contemplated.

A technology being developed that is related to virtual reality but moresophisticated is augmented reality. Augmented reality wearables (i.e.,headgear with an eye glasses form factor) will allow a user tosimultaneously view the real world and virtual, computer generatedcontent that is superimposed on the real world. To improve the illusionthat virtual content is real and/or to more seamlessly integrate thevirtual content and the real world it would be desirable that thevirtual content appear to exist in the inertial reference frame fixed tothe real environment of the user notwithstanding a user rotating theirhead with the headgear, or ambulating within their environment. So forexample if the virtual content were to include a virtual book resting ona corner of a real world desk, the book should remain fixed in the realworld, e.g., on the corner of the desk, even when the user rotates theirhead along with the augmented reality headgear that is projecting theimage of the book on the corner of the desk. To accomplish this themovements of the headgear carried on the user's head will need to becarefully tracked and the image being generated by the augmented realityheadgear rapidly shifted in the field of view to null out the effect ofthe movements of the headgear. One way to track the orientation of anobject is to use one or more gyroscopes. However gyroscopes inherentlysuffer from drift and therefore attempting to fix virtual objects in thereal world based on gyroscope output would lead to the virtual objectsslowly drifting relative to the real world when they are intended tomaintain fixed positions relative to the real world.

SUMMARY

According to certain embodiments disclosed herein augmented realityheadgear is equipped with at least one laser and at least one opticalsensor array. Each laser emits a beam that is partially reflected from aroom surface (e.g., wall, ceiling or floor) creating a speckle patternthat is detected by the associated optical sensor array. The opticalsensor arrays can for example be of the type that may be used in digitalcameras, however in present application the 2D optical sensor arraysneed not be used to capture a focused image of an object, rather theymay be capturing a speckle patterned generated by a laser reflected froma room surface. The movement (in some cases appropriately termed the“optical flow”) of the speckle pattern across the sensor array(s) isused to calculate the movement of the augmented reality headgear in aninertial reference frame fixed to the room. Images displayed usingeyepieces of the augmented reality are shifted based on the calculatedmovement in order to maintain their position or velocity in the inertialreference frame fixed to the room.

One aspect of the disclosure is an augmented reality headgear including:at least one source of imagewise modulated light; at least onetransparent eyepiece configured to couple the imagewise modulated lightinto a user's eye while allowing the user to see the real world; atleast a first coherent light source aimed outward from the augmentedreality headgear in a first direction so as to project coherent light onat least one environmental surface; at least a first sensor arrayconfigured to receive light diffracted by and reflected from the atleast one environmental surface, the light forming a first specklepattern on the first sensor array; electronic circuitry coupled to thesource of imagewise modulated light and the first sensor array andconfigured to: operate the source of imagewise modulate light to displaya virtual object at a set of coordinates defined in an inertialreference frame fixed to a physical space occupied by a user wearing theaugmented reality headgear; receive a first copy of the first specklepattern at a first time; receive a second copy of the first specklepattern at a second time; determine a shift in the second copy of thefirst speckle pattern relative to the first copy of the first specklepattern; determine a motion of the augmented reality headgear within thephysical space occupied by the user based on the shift in the secondcopy of the first speckle pattern relative to the first copy of thefirst speckle pattern; and based on the motion of the augmented realityheadgear, adjust the imagewise modulated light to compensate for themotion of the augmented reality headgear and maintain the virtual objectat the set of coordinates defined in the inertial reference frame.Additionally a second sensor array configured to receive light reflectedfrom the at least one environment surface may be provided. Additionallythe augmented reality may also include a second coherent light sourceaimed outward from the augmented reality headgear in a second directionso as to project coherent light on the at least one environmentalsurface. Additionally the augmented reality headgear may also include aleast one pupil stop configured to substantially exclude light from thesecond coherent light source that is reflected by the at least oneenvironmental surface from reaching the first sensor array.Additionally, the augmented reality headgear may further include atleast one optical component configured to establish mutually exclusiveemission solid angle ranges of the first coherent light source and thesecond coherent light source. Additionally, the first sensor array maybe mounted so as to have a first field of view and the second sensorarray may be mounted so as to have a second field of view and the firstfield of view may partly overlap the second field of view.

One aspect of the disclosure is a method of sensing and distinguishingtranslation motions of a structure along a set of three independent axesand rotation of the structure about one of the set of three independentaxes which includes: providing at least one source of coherent lightthat emits light over a predetermined solid angle range and is coupledto the structure; providing a first 2D optical sensor array that iscoupled to the structure, the first 2D optical sensor array having afirst normal vector pointing in a first direction; providing a second 2Doptical sensor array that is coupled to the structure, the second 2Doptical sensor array having a second normal vector pointing in a seconddirection wherein the first normal vector and the second normal vectordefine a plane and the first normal vector is angled with respect to thesecond normal vector in the plane; using the at least one source ofcoherent illumination to illuminate a non-specular environmentalsurface, whereby a first speckle pattern is produced at the first 2Doptical sensor array and a second speckle pattern is produced at the 2Doptical sensor array; sensing translation of the structure along a firstof the set of three independent axes that includes a nonzero projectionon the plane that is between the first normal vector and the secondnormal vector by sensing a first optical flow of the first specklepattern on the first 2D optical sensor array and sensing a secondoptical flow of second speckle pattern on the second 2D optical sensorarray wherein the first optical flow and the second optical flow haveopposite direction projections on a difference vector between the firstnormal vector and the second normal vector; sensing translation of thestructure along a second of the set of three independent axes thatincludes a nonzero projection on the plane that is outside an angularrange between the first normal vector and the second normal vector bysensing a third optical flow of the first speckle pattern on the first2D optical sensor array and sensing a fourth optical flow of the secondspeckle pattern on the second 2D optical sensor array wherein the thirdoptical flow and the fourth optical flow have common directionprojections on the difference vector between the first normal vector andthe second normal vector; sensing translation of the structure along athird of the set of three independent axes that includes a nonzerocomponent perpendicular to the plane by sensing same sense verticaldirection optical flows of the first speckle pattern on the first 2Doptical sensor array and the second speckle pattern on the second 2Doptical sensor array; and sensing rotation of the structure about thefirst of the three independent axes by sensing opposite verticaldirection optical flows of the first speckle pattern on the first 2Doptical sensor array and the second speckle pattern on the second 2Doptical sensor array.

One aspect of the disclosure includes a method of sensing anddistinguishing translation motions of a structure along a set of threeindependent axes and rotation about the set of three independent axes,which includes: providing a first 2D optical sensor array coupled to thestructure, the first 2D optical sensor array having a first surfacenormal pointed in a first direction and a first field of view; providinga second 2D optical sensor array coupled to the structure, the second 2Doptical sensor array having a second surface normal pointed in a seconddirection and a second field of view; providing a third 2D opticalsensor array coupled to the structure, the third 2D optical sensor arrayhaving a third surface normal pointed in a third direction and a thirdfield of view; wherein the first direction, the second direction and thethird direction are independent; providing at least one coherent lightsource that projects light into the first field of view, the secondfield of view and the third field of view, wherein light diffracted byand reflected from nonspecular surrounding surfaces forms a firstspeckle pattern on the first 2D optical sensor array, a second specklepattern on the second 2D optical sensor array and a third specklepattern on the third 2D optical sensor array; sensing rotation about thefirst direction by using the second 2D optical sensor array to sensetranslation of the second speckle pattern in a direction that isazimuthal with respect to the first direction and using the third 2Doptical sensor array to sense translation of the third speckle patternin a direction that is azimuthal with respect to the first direction;sensing translation in the first direction by using the second 2Doptical sensor array to sense translation of the second speckle patternin the first direction and using the third 2D optical sensor array tosense translation of the third speckle pattern in the third direction;sensing rotation about the second direction by using the first 2Doptical sensor array to sense translation of the first speckle patternin a direction that is azimuthal with respect to the second directionand using the third 2D optical sensor array to sense translation of thethird speckle pattern in a direction that is azimuthal with respect tothe second direction; sensing translation in the second direction byusing the first 2D optical sensor array to sense translation of thefirst speckle pattern in the second direction and using the third 2Doptical sensor array to sense translation of the third speckle patternin the second direction; sensing rotation about the third direction byusing the second 2D optical sensor array to sense translation of thesecond speckle pattern in a direction that is azimuthal with respect tothe third direction and using the first 2D optical sensor array to sensetranslation of the first speckle pattern in a direction that isazimuthal with respect to the third direction; and sensing translationin the third direction by using the first 2d optical sensor array tosense translation of the first speckle pattern in the third directionand using the second 2d optical sensor array to sense translation of thesecond speckle pattern in the third direction.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings illustrate the design and utility of preferred embodimentsof the present invention, in which similar elements are referred to bycommon reference numerals. In order to better appreciate how theabove-recited and other advantages and objects of the present inventionsare obtained, a more particular description of the present inventionsbriefly described above will be rendered by reference to specificembodiments thereof, which are illustrated in the accompanying drawings.Understanding that these drawings depict only typical embodiments of theinvention and are not therefore to be considered limiting of its scope,the invention will be described and explained with additionalspecificity and detail through the use of the accompanying drawings inwhich:

FIG. 1 is a front view of a user's head wearing augmented realityheadgear according to an embodiment disclosed herein;

FIG. 2. is a schematic diagram of a motion detection system included inthe augmented reality headgear shown in FIG. 1;

FIG. 3 is a schematic diagram of a portion of the motion detectionsystem shown in FIG. 2 illustrating optical isolation between twosources of coherent light according to an embodiment;

FIGS. 4a-4f show a sequence of speckle patterns corresponding to asequence of translation steps;

FIG. 5 is a schematic diagram of a motion detection system included inaugmented reality headgear according to an alternative embodiment;

FIG. 6 is a perspective view of augmented reality headgear with aneyeglasses form factor according to another embodiment;

FIG. 7 is a schematic diagram of a motion detection system included inthe augmented reality headgear shown in FIG. 6;

FIG. 8 is a block diagram of an augmented reality headgear which isapplicable to the augmented reality headgear shown FIG. 1 and FIG. 6according to an embodiment;

FIG. 9 is flowchart of a method of operating the augmented realityheadgear shown in FIG. 1 according to an embodiment; and

FIG. 10 is a schematic illustration of an example of the use ofaugmented reality headgear according to an embodiment.

DETAILED DESCRIPTION

FIG. 1 is a front view of a user's head 102 wearing augmented realityheadgear 104 according to an embodiment. The augmented reality headgear104 has an eyeglasses form factor and includes a frame 106 in which aremounted a left (user's left) eyepiece 108 and a right eyepiece 110 whichtake the form of lenses (eyepieces) of the headgear 104. The eyepieces108, 110 can include surface relief reflective and transmissivediffraction gratings that control the redirection of light toward theuser's eyes or, can include tilted, partially reflective mirrors,embedded in the bulk eyepieces 108, 110. For example each eyepiece 108,110 can include an incoupling grating (ICG) which receive imagewisemodulated light and diffracts the imagewise modulated light to anglesabove the critical angle for total internal reflection (TIR) within eacheyepiece. In addition to the ICG each eyepiece can further include anorthogonal pupil expansion grating (OPE) which incrementally deflectslight directed by the ICG to the OPE down toward an exit pupil expander(EPE) which incrementally deflects light toward out toward a user eyeposition. Alternatively, the eyepieces 108, 110 can include a partiallyreflective front surface coating that redirects light to the user'seyes. In the latter case the eyepieces need not be planar but caninclude curved front and back (near eye) surfaces that have finiteoptical power. Referring again to FIG. 1, a left source of imagewisemodulated light 112 is optically coupled to the left eyepiece 108 and aright source of imagewise modulated light 114 is optically coupled tothe right eyepiece 110. The sources of imagewise modulated light 112,114 can for example include Liquid Crystal on Silicon (LCoS) imageprojectors, fiber scanners, or emissive (e.g., micro Light EmittingDiode micro Organic Light Emitting Diode) display panels. The eyepieces108, 110 serve to optically couple imagewise modulated light to theuser's eye. The eyepieces 108, 110 are also transparent thereby allowingthe user to view the real world. The sources of imagewise modulatedlight 112, 114 and the eyepieces 108, 110 are used to display imageswhich in the present context are termed “virtual content”. The virtualcontent augments the real world which is visible through the eyepieces108, 110. The left eyepiece 108 in combination with the left source ofimagewise modulated light 112 together form a left display and the righteyepiece 110 in combination with the right source of imagewise modulatedlight 114 forms a right display.

For many applications it is desirable have the virtual contentpositioned or moving (e.g. a virtual person walking) in an inertialreference frame fixed to the environment (e.g., room) within which theuser of the augmented reality headgear is located, notwithstanding thefact that the augmented reality headgear 104 is being moved and rotatedwith the user as the user ambulates about the environment and turns hisor her head 102 to look in different directions. To achieve the latterobjective the imagery displayed via the eyepieces 108, 110 and thesources of imagewise modulated light 112, 114 must be shifted tocompensate for the user's motions. Determining the correct shiftrequires carefully tracking the user's motions (translations androtations). To this end, the augmented reality headgear 104 is equippedwith upward pointing laser (source of coherent illumination) 116, asideways pointing laser 118 and front pointing laser 120 all of whichare mechanically coupled to the frame 106. The pointing directions ofthe lasers 116, 118, 120 can also be altered relative to the foregoingdirections which are merely simple examples of suitable directions. Anupward facing 2D optical sensor array 122, a sideways facing 2D opticalsensor array 124 and a front facing 2D optical sensor array 126 are alsoprovided and are mechanically coupled to the frame 106. The 2D opticalsensor arrays 122, 124, 126 can for example comprise Complementary MetalOxide Semiconductor (CMOS) pixel arrays or Charge Couple Device (CCD)pixel arrays._Light emitted by the upward pointing laser 116 isangularly within a field of view of the upward facing 2D optical sensorarray 122, light emitted by the sideways pointing laser 118 is angularlywithin a field of view of the sideways facing 2D optical sensor array124 and light emitted by the front pointing laser 120 is angularlywithin a field of view of the front facing 2D optical sensor array 126.Although light from each laser 116, 118, 120 is angularly within thefield of view of a particular one of the 2D optical sensor arrays 122,124, 126 to the extent that the light is propagating away from theemitting laser and associated 2D optical sensor array 122, 124, 126, itwill not, to any significant degree, be detected by the associated 2Doptical sensor array 122, 124, 126 unless scattered back by a surface,e.g., a wall, a ceiling, or furniture on which it impinges. With theexception of windows and mirrors such surfaces are generally diffuse sothat the back scattered light will assume the form of a speckle patternthat fills the space between the scattering surface and 2D opticalsensor arrays 122, 124, 126. The speckle pattern, in this case, is thediffraction pattern of the small scale roughness of the surface. Thespeckle patterns are detected by the 2D optical sensor arrays 122, 124,126. Moreover as the user moves his head 102 with the augmented realityheadgear 104 the 2D optical sensor arrays 122, 124, 126 will movethrough the space filling speckle pattern and such movement isdetectable by reading out the 2D optical sensor arrays 122, 124, 126 atsuccessive times. Because the lasers 116, 118, 120 move along with the2D optical sensor arrays 122, 124, 126, the detected movement of thespeckle pattern across the 2D optical sensor arrays will be twice thephysical movement of the 2D optical sensor arrays 122, 124, 126. In thecase of rigid body mechanics there are three translation degrees offreedom (e.g., translation along Cartesian X, Y, Z axes) and threerotation degrees of freedom e.g., yaw, pitch an roll. For convenience wecan alternatively refer to rotations about the X, Y, and Z axes usingthe variables Rx, Ry, Rz.

FIG. 2 is a schematic diagram of a motion detection system 200 includedin the augmented reality headgear shown in FIG. 1. The motion detectionsystem 200 includes the upward pointing laser 116, the sideways pointinglaser 118, the front pointing laser 120, the upward facing 2D opticalsensor array 122, the sideways facing 2D optical sensor array 124 andthe front facing 2D optical sensor array 126. The aforementionedcomponents are shown oriented relative to a 3D Cartesian coordinatesystem 202 including Z (upward), Y (sideways, user's left) and X (front)axes. Coherent light from upward (+Z) facing laser 116 is incident on aceiling surface 204 which is non-specular and includes some small scalesurface roughness which diffracts the coherent light forming a specklepattern which impinges on the upward (+Z) facing 2D optical sensor array122. Similarly coherent light from the sideways (+Y) facing laser 118 isincident on a sideways facing patch of wall 206 which scatters lightforming a speckle pattern on the sideways (+Y) facing 2D optical sensorarray 124. Likewise coherent light from the front (+X) facing laser 120is incident on the rearward facing patch of wall 208 which scatterslight forming a speckle pattern on the front (+X) facing 2D opticalsensor array 126. In FIG. 2 rotations about the X, Y and Z axes areindicated by the notation Rx, Ry, Rz. A component of translation motionof the augmented reality headgear 104 along a particular Cartesian axes(X, Y, or Z) causes a speck shift parallel to the component on the two2D optical sensor arrays 122, 124 and/or 126 facing in directionsdifferent from the component of translation motion. Thus, for example atranslation component in the +X direction causes a +X directiontranslations of the speck patterns on the upward (+Z) facing 2D opticalsensor array 122 and on the sideways (+Y) facing 2D optical sensor array124. The induced translations of the speckle patterns are equal to twicethe physical translations of the headgear 104. Each 2D optical sensorarray 122, 124, 126 is operated at a frame rate that is sufficientlyhigh so that a laser spot that is being sensed by the 2D sensor array122, 124, 126 moves a distance that is a fraction of the size, (i.e.,the Full Width Half Max, FWHM) of the spot between successive frames.

As shown in the sche 101782-006810US-1091827 atic representation in FIG.2 the 2D optical sensor arrays 122, 124, 126 are displaced from theCartesian coordinate system origin. In the augmented reality headgear104, the 2D optical sensor arrays are displaced from the effectivecenter of rotation for head movements which is located in the vicinityof the back of the user's neck. Rotations about the X, Y and Z axes aresensed by sensing shifts of the speckle patterns produced on the 2Doptical sensor arrays 122, 124, 126. A component of rotation about agiven axes (X, Y or Z) will induce a speckle shift that is azimuthalwith respect to the given axes on the 2D optical sensor arrays 122, 124or 126 other than the 2D optical sensor array 122, 124 or 126 facing ina direction parallel to the given axis. (For the present purposes thepositive sense of rotation about the positive Cartesian axes is definedusing the right hand rule.) Thus for example the −X direction isazimuthal with respect to the Z-axis on the sideways (+Y) facing 2Doptical sensor array 124 and the +Y direction is also azimuthal withrespect to the Z-axis on the front (+X) facing 2D optical sensor array126. Accordingly a positive Rz rotation component about the Z-axis willinduce a X direction speckle shift on the sideways (+Y) facing 2Doptical sensor array 124 and will induce a −Y direction speck shift onthe front (+X) facing 2D optical sensor array 126. Matrix equation EQU.1 below relates incremental translations along Cartesian X, Y and Z axesand incremental rotations about the Cartesian X, Y and Z axes to shiftsof the speckle patterns on the 2D optical sensor arrays 122, 124, 126.

$\begin{matrix}{{\begin{bmatrix}0 & C_{y,x,y} & 0 & 0 & 0 & C_{{Rz},x,y} \\0 & 0 & C_{z,x,z} & 0 & {- C_{{Ry},x,z}} & 0 \\C_{x,y,x} & 0 & 0 & 0 & 0 & {- C_{z,y,x}} \\0 & 0 & C_{z,y,x} & C_{{Rx},y,z} & 0 & 0 \\C_{x,z,x} & 0 & 0 & 0 & C_{{Ry},z,x} & 0 \\0 & C_{y,z,y} & 0 & {- C_{{Rx},z,y}} & 0 & 0\end{bmatrix} \cdot \begin{bmatrix}{\Delta\; x} \\{\Delta\; y} \\{\Delta\; z} \\{\Delta\;{Rx}} \\{\Delta\;{Ry}} \\{\Delta\;{Rz}}\end{bmatrix}} = \begin{bmatrix}S_{x}^{y} \\S_{x}^{z} \\S_{y}^{x} \\S_{y}^{z} \\S_{z}^{x} \\S_{z}^{y}\end{bmatrix}} & {{EQU}.\mspace{14mu} 1}\end{matrix}$

The nonzero coefficients in the coefficient matrix on the left hand sideof EQU. 1 have three subscripts. The first subscript identifies one ofsix degrees of freedom on motion of the augmented reality headgear 104among X, Y, Z translations and Rx, Ry, Rz rotations about the X, Y and Zaxes. The second subscript identifies one of the 2D optical sensorarrays 122, 124, 126 by the direction (X, Y, or Z) in which it faceswhich is equal to the direction of the normal vector to the front (lightreceiving) surface of the 2D optical sensor array 122, 124, 126. Thethird subscript identifies a direction (X, Y, or Z) of speckle shift onthe particular 2D optical sensor array 122, 124, 126 identified by thesecond subscript. The non-zero coefficients in the first three columnsof the coefficient matrix which relate to translation degrees of freedomhave values of 2. The non-zero coefficients in the third through sixthcolumns of the translation matrix have values of 2/R_(sensor), whereR_(sensor) is the distance between the 2D optical sensor array 122, 124or 126 identified by the second subscript and the effective center ofrotation (e.g., back of user's neck) when the user is wearing theaugmented reality headgear 104.

The column vector on the left side of EQU. 1 includes incrementaltranslations Δx, Δy, Δz as the first three elements and incrementalrotations ΔRx, ΔRy, ΔRz about the X, Y and Z axes as the last threeelements. Each element of column vector on the right hand side of EQU. 1is a speckle shift on one of the 2D optical sensor arrays 122, 124, 126.Each speckle shift element is denoted by a subscripted and superscriptedletter S. The subscript identifies one of the 2D optical sensor arrays122, 124, 126 by the direction in which its normal vector is oriented(the direction it faces). The superscript identifies a direction ofspeckle shift on the 2D optical sensor array 122, 124, 126. By way ofillustration the first row of the coefficient matrix C indicates thatboth a translation in the y direction (as indicated by C_(y,x,y)) and arotation about the Z axis (as indicated by C_(Rz,x,y)) will cause a ydirection (azimuthal with respect to z axis) speckle shift on the front(+X) direction facing 2D optical sensor array 126). The coefficientmatrix is readily invertible leading to matrix equation EQU. 2 givenbelow:

$\begin{matrix}{{\left\lbrack C^{- 1} \right\rbrack \cdot \begin{bmatrix}S_{x}^{y} \\S_{x}^{z} \\S_{y}^{x} \\S_{y}^{z} \\S_{z}^{x} \\S_{z}^{y}\end{bmatrix}} = \begin{bmatrix}{\Delta\; x} \\{\Delta\; y} \\{\Delta\; x} \\{\Delta\;{Rx}} \\{\Delta\; R\; y} \\{\Delta\;{Rz}}\end{bmatrix}} & {{EQU}.\mspace{14mu} 2}\end{matrix}$

where C⁻¹ is the inverse of the matrix in EQU. 1. EQU. 2 is used todetermine incremental translations and rotations of the augmentedreality headgear 104 based on the vector of speckle shifts appearing inthe left hand side of EQU. 2. The incremental speckle shifts can beobtained by reading the speckle patterns formed on the 2D optical sensorarrays 122, 124, 126 at two successive times and determining therelative shift of the speckle patterns at the two successive times.Optical flow tracking methods such as the Farneback method or normalizedcross correlation may be used to determine the relative shift of thespeckle patterns. Based on the determined incremental translations androtations, the virtual content being output via the sources of imagewisemodulated light 112, 114 and the eyepieces 108, 110 is adjusted tomaintain position and/or motion in an inertial coordinate system fixedto the environment within which the augmented reality headgear 104 isbeing used. The 2D optical sensor arrays 122, 124, 126 are read at asufficiently high rate compared to the maximum anticipated rate of theaugmented reality headgear, so that frame-to-frame change in thepointing angle of the lasers 116, 118, 120 is fraction of the FWHM beamdivergence of the lasers 116, 118, 120 (including the effect of thediffusers 310, 312 FIG. 3). Accordingly, frame-to-frame change inspeckle patterns sensed by the 2D optical sensor arrays 122, 124, 126 isprimarily a shift (optical flow). Additionally, the lasers 116, 118, 120can be operated in pulse mode, at a pulse rate equal to a frame rate ofthe 2D optical sensor arrays 122, 124, 126 and a pulse width that issubstantially shorter than the frame period (1/frame rate) of the 2Doptical sensor arrays 122, 124, 126. Using such a short laser pulsewidth will help to avoid motion induced blur of the speckle pattern.

FIG. 3 is a schematic diagram of a portion of the motion detectionsystem shown in FIG. 2 illustrating optical isolation between twosources of coherent light according to an embodiment. The view shown inFIG. 3 includes the sideways pointing laser 118 and the front pointinglaser 120. As shown the sideways pointing laser 118 includes a firstlaser diode 302 optically coupled to a first collimating lens 304 andsimilarly the front pointing laser 120 includes a second laser diode 306optically coupled to a second collimating lens 308. The collimatinglenses 304, 308 establish mutually exclusive solid angle ranges ofcoverage (emission) of the sideways pointing laser 118, and the frontpointing laser 120. The first collimating lens 304 forms a sidewayspropagating light beam and the second collimating lens 308 forms afrontward propagating light beam. Alternatively, in lieu of collimatinglenses, lenses that form controlled divergence beams may be used.Furthermore beam shaping lenses that establish a certain radial (e.g.,flattop) or nonaxisymmetric beam profile may be user in lieu of thecollimating lenses. A first low angle diffuser 310 is positioned infront of the first collimating lens 304 and a second low angle diffuser312 is positioned in front of the second collimating lens 308. The lowangle diffusers 310, 312 reduce the luminance of light beams formed bythe collimating lenses 304, 308 and are useful for eye safety. Note thatthe laser diodes 302, 306 may be infrared emitting in which case theuser would not be able to see the emitted light. The low angle diffusers310, 312 can, by way of non-limiting example, be characterized by adiffusion FWHM of 2° to 20°. The sideways facing 2D optical sensor array124 and the front facing 2D optical sensor array 126 are also shown inFIG. 3. A sideways facing pupil stop 314 is positioned in front of thesideways facing 2D optical sensor array 124 and serves to limit thefield of view of the sideways facing 2D optical sensor array 124.Similarly a front facing pupil stop 316 is positioned in front of thefront facing 2D optical sensor array 126 and serves to limit the fieldof view of the front facing 2D optical sensor array 126. The sidewaysfacing pupil stop 314 establishes a field of view of the sideways facing2D optical sensor array 124 that substantially overlaps a solid anglerange of emission of the sideways pointing laser 118 as expanded by thefirst low angle diffuser 310 and substantially excludes a solid anglerange emission of the front pointing laser 120 as expanded by the secondlow angle diffuser 312. Similarly the front facing pupil stop 316establishes a field of view of the front facing 2D optical sensor array126 that substantially overlaps a solid angle range of emission of thefront pointing laser 120 as expanded by the second low angle diffuser312 and substantially excludes a solid angle range of emission of thesideways pointing laser 118 as expanded by the first low angle diffuser310. Thus each 2D optical sensor array will receive only a singlespeckle pattern produced by light emitted by its associated laser.Another purpose of the pupil stops 314, 316 is enlarge the size of thespeckles in the speckle pattern that is incident on each 2D opticalsensor array 124, 126. The characteristic size of the speckles in thespeck pattern should be equal to or larger than the size of individualsensor elements (pixels) that make up the 2D sensor arrays 124, 126.Although not shown in FIG. 3 the upward pointing laser 116 can have thesame internal design as that shown in FIG. 3 for the sideways pointinglaser 118 and the front pointing laser 120 and also be equipped with alow angle diffuser, and the upward facing 2D optical sensor array 122can also be equipped with a pupil stop.

According to an alternative embodiment the sensor arrays 122, 124, 126are spectrally isolated from emissions of the lasers 116, 118, 120 otherthan the one with which they are associated. In one implementation theupward pointing laser 116 emits a first spectral line having a firstpeak wavelength, the sideways pointing laser 118 emits a second spectralline having a second peak wavelength and the front pointing laser 120emit a third spectral line having a third peak wavelength. A firstspectrally selective filter that transmits the first spectral line butnot the second spectral line or the third spectral line is positionedover the upward facing 2D optical sensor array 122; a second spectrallyselective filter that transmits the second spectral line but not thefirst spectral line or the third spectral line is positioned over thesideways facing 2D optical sensor array 124; and a third spectrallyselective filter that transmits the third spectral line but not thefirst spectral line or the second spectral line is positioned over thefront facing 2D optical sensor array 126.

FIG. 4a is an initial image of a speckle pattern 400 that can be used totrack motion in the system shown in FIGS. 1-3. FIGS. 4b-4f show asequence of images 408, 410, 412, 414, 416 of the speckle pattern 400shown in FIG. 4a after a sequence of translation increments. An initialarbitrary position 402 in the speckle pattern 400 is marked with avertical line 404 and a horizontal line 406. An optical flow trackingmethod, such as, but not limited to those mentioned above can be used totrack the movement of the speckle pattern. The translation distance ofthe speckle pattern is equal to twice the physical translation augmentedreality headgear 104 which includes the motion detection system 200. Thelocation of the arbitrarily selected position 402 is shown in each ofthe successive FIGS. 4b -4 f.

FIG. 5 is a schematic diagram of a motion detection system 500 includedin augmented reality headgear 104 according to an alternativeembodiment. The system 300 includes a first laser 506 oriented to pointin a first direction, a second laser 508 oriented to point in a seconddirection, and a third laser oriented to point in third direction. Thesystem 300 also includes a first 2D optical sensor array facing in thefirst direction, a second 2D optical sensor array facing in the seconddirection and a third 2D optical sensor array facing in the thirddirection. Whereas, in the case of the motion detection system 300 shownin FIG. 3 the lasers 116, 118, 120 point in a set of three orthogonaldirections and the 2D optical sensor arrays 122, 124, 126 are orientedto face in the same set of three orthogonal directions, in the case ofthe motion detection system 500 shown in FIG. 5, the first, second andthird directions are not orthogonal.

FIG. 6 is a perspective view of an augmented reality headgear 600 withan eyeglasses form factor according to another embodiment and FIG. 7 isa schematic representation of a motion detection system 700 that isincluded in the augmented reality headgear 600. The augmented realityheadgear 600 includes a frame 602 supporting a left (user's left)transparent eyepiece 604 and a right transparent eyepiece 606. The lefttransparent eyepiece 604 includes a left ICG 604A, a left OPE 604B and aleft EPE 604C and similarly the right transparent eyepiece 606 includesa right ICG 606A, a right OPE 606B and a right EPE 606C. Each of theeyepieces 604, 606 can alternatively include multiple waveguides tohandle multiple color channels and or to output imagery at differentwavefront curvatures (corresponding to different virtual imagedistances). A left source of imagewise modulated light 608 is opticallycoupled to the left transparent eyepiece 604 and right source ofimagewise modulated light 610 is optically coupled to the righttransparent eyepiece 606. The transparent eyepieces 604, 606 serve tooptically couple imagewise modulated light to a user's eyes.

The augmented reality headgear 600 is further equipped with a rightlaser 611, a left laser 612, a left 2D optical sensor array 614 and aright 2D optical sensor array 616. The forward direction corresponds tothe +X axis of the Cartesian coordinate system triad shown in FIG. 6.The left laser 612 and the left 2D optical sensor array 614 face in adirection that is tilted to the left azimuthally (rotated about thevertically oriented Z axis) with respect to the forward direction andright laser 611 and the right 2D optical sensor array 616 face in adirection that is tilted to the right azimuthally with respect to theforward direction. The direction in which the left laser 612 and leftoptical sensor 614 face is defined by a left normal vector N_(L) that isnormal to a light receiving surface of the left optical sensor 614 andthe direction in which the right laser 611 and the right optical sensor616 face is defined by a right normal vector N_(R) that is normal to alight receiving surface of the right optical sensor 616. Alternativelythere can be a difference between the direction in which each laser andassociated sensor face. A difference vector D between the left normalvector N_(L) and the right normal vector N_(R) is shown in FIG. 7. Theleft 2D optical sensor array 614 and the right 2D optical sensor array616 have separate fields of view. The field of view of the left 2Doptical sensor array 614 includes, at least, a substantial portion ofthe range of emission of the left laser 612 and the field of view of theright 2D optical sensor array 616 includes, at least, a substantialportion of the solid angle range of emission of the right laser 611. Thefields of view of the left and right 2D optical sensor arrays 614, 616could be limited by pupil stops or other field of view limiting opticalcomponents (not shown in FIG. 6). For example, the field of view of theleft 2D optical sensor array 614 could be limit to exclude the angularrange of emission right laser 611 and vice versa.

Optionally the left 2D optical sensor array 614 can be equipped with aleft imaging lens 618 and the right 2D optical sensor array 616 can beequipped with a right imaging lens 620. The imaging lenses 618, 620focus and magnify or demagnify speckle light from, respectively a leftfocal plane 622 and a right focal plane 624 that are positioned in spacein front of the imaging lenses 618, 620 onto the 2D optical sensorarrays 614, 616.

It should be noted that the Y axis of the Cartesian coordinate triadextends sideways from left to right. The motion detection system 700incorporated in the augmented reality headgear 600 is capable of sensingand discriminating 4 degrees of freedom, including translationcomponents along the X-axis, Y-axis and Z-axis and rotation about theforward facing X-axis. The left normal vector N_(L) and the right normalvector N_(R) define a virtual plane, and on the plane defined by N_(L)and N_(R) there is an angular range between N_(L) and N_(R). Atranslation component having a projection on the plane defined by N_(L)and N_(R) in the angular range between N_(L) and N_(R) can be sensed bysensing opposite direction speckle pattern shifts on the left and right2D optical sensor arrays 614, 616 that are parallel to the plane definedby N_(L) and N_(R). In the preceding case a first optical flow on theleft 2D optical sensor array 614 and a second optical flow on the right2D optical sensor array 616 have opposite direction projections on thedifference vector D.

On the other hand, a translation component having a projection on theplane defined by N_(L) and N_(R) outside the angular range between N_(L)and N_(R) can be sensed by sensing same direction speckle pattern shiftson the left and right 2D optical sensor arrays 614, 616 that areparallel to the plane defined by N_(L) and N_(R). In the latter case afirst optical flow on the left 2D optical sensor array 614 and a secondoptical flow on the right 2D optical sensor array 616 have commondirection projections on the difference vector D. Furthermore atranslation component perpendicular to the plane defined by N_(L) andN_(R) can be sensed by sensing same direction speckle pattern shifts onthe left and right 2D optical sensor arrays 614, 616 that areperpendicular to the plane defined by N_(L) and N_(R). Additionallyrotations about the forward extending +X axis can be sensed by sensingopposite vertical direction speckle pattern shifts on the left and right2D optical sensor arrays 614, 616 that are perpendicular to the planedefined by N_(L) and N_(R).

FIG. 8 is a block diagram of augmented reality headgear 900 according toan embodiment. The design shown in FIG. 8 can be used for the augmentedreality headgear shown in FIG. 1 and FIG. 6 according to certainembodiments. The augmented reality headgear 900 includes a left lighttransmissive (see-through) eyepiece 902, a right light transmissiveeyepiece 904, a first laser diode 906, a first 2D optical sensor array908, an N^(TH) laser diode 910 and an N^(TH) 2D optical sensor array 912all mechanically coupled to an augmented reality headgear frame 914. Itshould be understood that the use of the identifier “N^(TH)” signifiesthat the number of like components including the one identified by asthe “N^(TH)” is variable. For example in the case of the augmentedreality headgear 104 shown in FIG. 1 three laser diodes are utilizedwithin the three lasers 116, 118, 120, whereas in the case of theaugmented reality headgear 600 shown in FIG. 6 a single laser diode (notshown in FIG. 6) is provided within the forward directed laser 612.

A left source of imagewise modulated light 916 is optically coupled tothe left light transmissive eyepiece 902 and a right source of imagewisemodulated light 918 is optically coupled to the right light transmissiveeyepiece 904. The sources of imagewise modulated light 916, 918 can forexample comprise fiber scanners, LCoS projectors or MEMS light beamscanners, or micro emissive displays. A left image data source 920 iscoupled to the left source of imagewise modulated light 916 and a rightimage data source 922 is coupled to right source of imagewise modulatedlight 918. The image data sources 920, 922 can, for example, take theform of display drivers. The left source of imagewise modulated light916 in combination with the left light transmissive eyepiece 902 forms aleft display 948 and the right source of imagewise modulated light 918in combination with the right light transmissive eyepiece 904 forms aright display 950. The left and right sources of imagewise modulatedlight 916, 918 modulate light in accordance with data provided by,respectively, the left and right image data sources 920, 922. The leftand right image data sources 920, 922 can take the form of frame bufferssupplied with data by a graphics processing unit (GPU) that along with amicroprocessor runs a game engine program.

The first laser diode 906 is optically coupled through a firstcollimating lens 924 to a first diffuser 926 and the N^(TH) laser diode912 is optically coupled through an N^(TH) collimating lens 928 to anN^(TH) diffuser 930. Coherent light from the first laser diode 906 thatis coupled through the first collimating lens 924 and the first diffuser926 is incident on a first surface patch 932 (e.g., wall, ceiling,floor, furniture) in an environment of the augmented reality headgear900 which forms a first speckle pattern (diffraction pattern of smallscale roughness of surface) which is incident on the first 2D opticalsensor array 908. Similarly, coherent light from the N^(TH) laser diode910 that is coupled though the N^(TH) collimating lens 928 and theN^(TH) diffuser 930 is incident on an N^(TH) surface patch 934 in theenvironment of the augmented reality headgear 900 forming an N^(TH)speckle pattern which is incident on the N^(TH) 2D optical sensor array912.

A first sensor readout circuit 936 is coupled to the first 2D opticalsensor array 908 and an N^(TH) sensor readout circuit 938 is coupled tothe N^(TH) 2D optical sensor array 912. An inertial measurement unit(IMU) 952 is mechanically coupled to the frame 914. The first sensorreadout circuit 936, the N^(TH) sensor readout circuit 938, the leftimage data source 920, the firth image data source 922, the IMU 952, atleast one processor 940, at least one program memory 942 and at leastone workspace memory 944 are coupled together via at least one bus 946.The at least one processor 940 can for example include a microprocessor,a graphics processing unit, a digital signal processor and/or amicrocontroller. The IMU 952 can be used in conjunction with thecomponents described above which detect motion via speckle optical flow.For example the IMU 952 can be used as an additional redundant source ofmotion information to improve accuracy or information from the IMU canbe combined with information obtained via speckle flow monitoring tofully determine the 6DoF of the headgear 900.

FIG. 9 is flowchart of a method 1000 of operating the augmented realityheadgear 104 shown in FIG. 1 according to an embodiment. The augmentedreality headgear 600, 900 shown in FIGS. 6, 8 can be operated in ananalogous manner. In block 1002 surfaces (e.g., 204, 206, 208) in alocal environment of the headgear 104 are illuminated with at lasers116, 118, 120 fixed to the augmented reality headgear 104. In block 1004the displays of the augmented reality headgear is operated to display atleast one virtual object at a position (defined by a set of coordinates)in the local environment. The eyepieces 108, 110 in combination with thesources of imagewise modulated light 112, 114 together form displays.FIG. 10 is a schematic illustration of an example of the use of theaugmented reality 104 headgear shown in FIG. 1 being used according tothe method shown in FIG. 10. In FIG. 10 the headgear 1100 is shown onthe user's head 102. The user is viewing a virtual object 1102 in theform of a book which is displayed using the displays of the augmentedreality headgear. The virtual object 1102 located on a real table 1104.Other examples of virtual content can include, for example, people,animals, imaginary creatures, and/or moving objects. In the case ofmoving virtual content, the movement of the virtual content is definedin an inertial reference frame fixed to the physical environment and themotion of the head gear is tracked so that it can be compensated for(nulled out) so that the movement of the head gear does add velocity tothe intended movement of the virtual content relative to the physicalenvironment. Referring again to FIG. 9 in block 1006 the light sensorarrays 122, 124, 126 are used to sense speckle patterns produced by thelasers 116, 118, 120 being scattered by the environment surfaces (e.g.,204, 206, 208). In executing block 1006 and block 1010 described below,the speckle patterns may, for example, be received from the sensorreadout circuits 936, 938 (FIG. 8) in the workspace memory 944 (FIG. 8)under the control of the at least one processor 940 (FIG. 8). Block 1008marks the start of a loop that is executed for each of a succession oftimes. In block 1010 the light sensor arrays are again used to sense thespeckle patterns produced by the lasers 116, 118, 120 being scattered bythe environment surfaces (e.g., 204, 206, 208). In block 1012 the shiftsin each particular speckle pattern compared to at least one precedingmeasurement of the particular speckle patterns are determined. Anoptical flow determination method such as discussed above may be used inexecuting block 1012. In block 1014 changes in the translation androtation coordinates of the augmented reality head gear are determinedbased on the shifts in the speckle patterns determine in block 1012. Inexecuting block 1014 EQU. 2 described above can be used. Next in block1016 the position (coordinates) of the virtual object on the display ofthe augmented reality headgear 104 is shifted in accordance with thechange of at least one coordinate, as determined in block 1014, in orderto maintain the position of the virtual object in the local environment,for example to maintain the virtual book 1102 in position on the realtable 1104. The position of the virtual object is shifted by adjustingthe imagewise modulated light that is coupled to the user's eyes throughthe eyepieces. The method 1100 can be executed under the control of aprogram that is stored in the at least one memory 942 and executed bythe at least one processor 940 using the at least one workspace memory944. More generally speaking, the head gear 100, 600, 900 is providewith some form of electronic circuitry, which may alternatively include,by way of nonlimitive example, an Application Specific IntegratedCircuit (ASIC) and/or FPGA that determines the speckle shifts betweensuccessive times and based on the speckle shifts determines incrementalmotions (translations and rotations) between successive times, andadjust the position of displayed virtual objects to compensate for theincremental motions.

While embodiments described above include augmented reality glasses thatinclude transparent eyepieces through which the user may view the realworld while also viewing virtual content, alternatively the 6DoFtracking systems described above may be incorporated in virtual realitygoggles in which the user's view of the real world is occluded and theuser may only see virtual content. The 6DoF systems described above mayalso be applied to a type of augmented reality in which the user cannotdirectly view the real world but can view imagery of the real worldcaptured by one or more cameras and displayed to the user along withvirtual content.

Various example embodiments of the invention are described herein.Reference is made to these examples in a non-limiting sense. They areprovided to illustrate more broadly applicable aspects of the invention.Various changes may be made to the invention described and equivalentsmay be substituted without departing from the true spirit and scope ofthe invention. In addition, many modifications may be made to adapt aparticular situation, material, composition of matter, process, processact(s) or step(s) to the objective(s), spirit or scope of the presentinvention. Further, as will be appreciated by those with skill in theart that each of the individual variations described and illustratedherein has discrete components and features which may be readilyseparated from or combined with the features of any of the other severalembodiments without departing from the scope or spirit of the presentinventions. All such modifications are intended to be within the scopeof claims associated with this disclosure.

The invention includes methods that may be performed using the subjectdevices. The methods may comprise the act of providing such a suitabledevice. Such provision may be performed by the end user. In other words,the “providing” act merely requires the end user obtain, access,approach, position, set-up, activate, power-up or otherwise act toprovide the requisite device in the subject method. Methods recitedherein may be carried out in any order of the recited events which islogically possible, as well as in the recited order of events.

Example aspects of the invention, together with details regardingmaterial selection and manufacture have been set forth above. As forother details of the present invention, these may be appreciated inconnection with the above-referenced patents and publications as well asgenerally known or appreciated by those with skill in the art. The samemay hold true with respect to method-based aspects of the invention interms of additional acts as commonly or logically employed.

In addition, though the invention has been described in reference toseveral examples optionally incorporating various features, theinvention is not to be limited to that which is described or indicatedas contemplated with respect to each variation of the invention. Variouschanges may be made to the invention described and equivalents (whetherrecited herein or not included for the sake of some brevity) may besubstituted without departing from the true spirit and scope of theinvention. In addition, where a range of values is provided, it isunderstood that every intervening value, between the upper and lowerlimit of that range and any other stated or intervening value in thatstated range, is encompassed within the invention.

Also, it is contemplated that any optional feature of the inventivevariations described may be set forth and claimed independently, or incombination with any one or more of the features described herein.Reference to a singular item, includes the possibility that there areplural of the same items present. More specifically, as used herein andin claims associated hereto, the singular forms “a,” “an,” “said,” and“the” include plural referents unless the specifically stated otherwise.In other words, use of the articles allow for “at least one” of thesubject item in the description above as well as claims associated withthis disclosure. It is further noted that such claims may be drafted toexclude any optional element. As such, this statement is intended toserve as antecedent basis for use of such exclusive terminology as“solely,” “only” and the like in connection with the recitation of claimelements, or use of a “negative” limitation.

Without the use of such exclusive terminology, the term “comprising” inclaims associated with this disclosure shall allow for the inclusion ofany additional element—irrespective of whether a given number ofelements are enumerated in such claims, or the addition of a featurecould be regarded as transforming the nature of an element set forth insuch claims. Except as specifically defined herein, all technical andscientific terms used herein are to be given as broad a commonlyunderstood meaning as possible while maintaining claim validity.

The breadth of the present invention is not to be limited to theexamples provided and/or the subject specification, but rather only bythe scope of claim language associated with this disclosure.

What is claimed:
 1. Augmented reality headgear comprising: at least onesource of imagewise modulated light configured to provide an imagewisemodulated light; at least one transparent eyepiece configured to couplethe imagewise modulated light into a user's eye while allowing a user tosee a real world; at least a first coherent light source aimed outwardfrom the augmented reality headgear in a first direction so as toproject coherent light on at least one environmental surface; at least afirst sensor array configured to receive light reflected from the atleast one environmental surface, the light forming a first specklepattern on the first sensor array; and electronic circuitry coupled tothe source of imagewise modulated light and the first sensor array andconfigured to: operate the at least one source of imagewise modulatedlight to display a virtual object at a set of coordinates defined in aninertial reference frame fixed to a physical space occupied by the userwearing the augmented reality headgear; receive a first copy of thefirst speckle pattern at a first time; receive a second copy of thefirst speckle pattern at a second time; determine a shift in the secondcopy of the first speckle pattern relative to the first copy of thefirst speckle pattern; and determine a motion of the augmented realityheadgear within the physical space occupied by the user based on theshift in the second copy of the first speckle pattern relative to thefirst copy of the first speckle pattern.
 2. The augmented realityheadgear according to claim 1 wherein the electronic circuitry isfurther configured to: based on the motion of the augmented realityheadgear, adjust the imagewise modulated light to compensate for themotion of the augmented reality headgear and maintain the virtual objectat the set of coordinates defined in the inertial reference frame. 3.The augmented reality headgear according to claim 1 further comprising:a second sensor array configured to receive light from the at least oneenvironment surface.
 4. The augmented reality headgear according toclaim 3 further comprising a second coherent light source aimed outwardfrom the augmented reality headgear in a second direction so as toproject coherent light on the at least one environmental surface.
 5. Theaugmented reality headgear according to claim 4 further comprising aleast one aperture stop configured to substantially exclude light fromthe second coherent light source that is reflected by the at least oneenvironmental surface from reaching the first sensor array.
 6. Theaugmented reality headgear according to claim 4 further comprising atleast one optical component configured to establish mutually exclusiveemission solid angle ranges of the first coherent light source and thesecond coherent light source.
 7. A method of sensing and distinguishingtranslation motions of a structure along a set of three independent axesand rotation of the structure about one of the set of three independentaxes, the method comprising: providing at least one source of coherentlight that emits light over a predetermined solid angle range and iscoupled to the structure; providing a first 2D optical sensor array thatis coupled to the structure, the first 2D optical sensor array having afirst normal vector pointing in a first direction; providing a second 2Doptical sensor array that is coupled to the structure, the second 2Doptical sensor array having a second normal vector pointing in a seconddirection, wherein the first normal vector and the second normal vectordefine a plane and the first normal vector is angled with respect to thesecond normal vector in the plane; using the at least one source ofcoherent light to illuminate a non-specular environmental surface,whereby a first speckle pattern is produced at the first 2D opticalsensor array and a second speckle pattern is produced at the second 2Doptical sensor array; sensing translation of the structure along a firstof the set of three independent axes that includes a nonzero projectionon the plane that is between the first normal vector and the secondnormal vector by sensing a first optical flow of the first specklepattern on the first 2D optical sensor array and sensing a secondoptical flow of second speckle pattern on the second 2D optical sensorarray, wherein the first optical flow and the second optical flow haveopposite direction projections on a difference vector between the firstnormal vector and the second normal vector; sensing translation of thestructure along a second of the set of three independent axes thatincludes a nonzero projection on the plane that is outside an angularrange between the first normal vector and the second normal vector bysensing a third optical flow of the first speckle pattern on the first2D optical sensor array and sensing a fourth optical flow of the secondspeckle pattern on the second 2D optical sensor array, wherein the thirdoptical flow and the fourth optical flow have common directionprojections on the difference vector between the first normal vector andthe second normal vector; sensing translation of the structure along athird of the set of three independent axes that includes a nonzerocomponent perpendicular to the plane by sensing same sense verticaldirection optical flows of the first speckle pattern on the first 2Doptical sensor array and the second speckle pattern on the second 2Doptical sensor array; and sensing rotation of the structure about thefirst of the set of three independent axes by sensing opposite verticaldirection optical flows of the first speckle pattern on the first 2Doptical sensor array and the second speckle pattern on the second 2Doptical sensor array.
 8. A method of sensing and distinguishingtranslation motions of a structure along a set of three independent axesand rotation about the set of three independent axes, the methodcomprising: providing a first 2D optical sensor array coupled to thestructure, the first 2D optical sensor array having a first surfacenormal pointed in a first direction and a first field of view; providinga second 2D optical sensor array coupled to the structure, the second 2Doptical sensor array having a second surface normal pointed in a seconddirection and a second field of view; providing a third 2D opticalsensor array coupled to the structure, the third 2D optical sensor arrayhaving a third surface normal pointed in a third direction and a thirdfield of view; wherein the first direction, the second direction and thethird direction are independent; providing at least one coherent lightsource that projects light into the first field of view, the secondfield of view and the third field of view, wherein light reflected fromnonspecular surrounding surfaces forms a first speckle pattern on thefirst 2D optical sensor array, a second speckle pattern on the second 2Doptical sensor array, and a third speckle pattern on the third 2Doptical sensor array; sensing rotation about the first direction byusing the second 2D optical sensor array to sense translation of thesecond speckle pattern in a direction that is azimuthal with respect tothe first direction, and using the third 2D optical sensor array tosense translation of the third speckle pattern in a direction that isazimuthal with respect to the first direction; sensing translation inthe first direction by using the second 2D optical sensor array to sensetranslation of the second speckle pattern in the first direction, andusing the third 2D optical sensor array to sense translation of thethird speckle pattern in the first direction; sensing rotation about thesecond direction by using the first 2D optical sensor array to sensetranslation of the first speckle pattern in a direction that isazimuthal with respect to the second direction, and using the third 2Doptical sensor array to sense translation of the third speckle patternin a direction that is azimuthal with respect to the second direction;sensing translation in the second direction by using the first 2Doptical sensor array to sense translation of the first speckle patternin the second direction, and using the third 2D optical sensor array tosense translation of the third speckle pattern in the second direction;sensing rotation about the third direction by using the second 2Doptical sensor array to sense translation of the second speckle patternin a direction that is azimuthal with respect to the third direction,and using the first 2D optical sensor array to sense translation of thefirst speckle pattern in a direction that is azimuthal with respect tothe third direction; and sensing translation in the third direction byusing the first 2D optical sensor array to sense translation of thefirst speckle pattern in the third direction, and using the second 2Doptical sensor array to sense translation of the second speckle patternin the third direction.